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Ranking scholarly journals forms a major feature of the Excellence in Research for Australia (ERA) initiative. We argue this process is 
not only a flawed system of measurement, but more significantly erodes the very contexts that produce ‘quality’ research. We argue that 
collegiality, networks of international research, the socio-cultural role of the academic journal, as well as the way academics research in 
the digital era, are either ignored or negatively impacted upon by ranking exercises such as those posed by the ERA. 


It has recently been announced that the Excellence 
in Research for Australia (ERA) initiative will remain 
largely unchanged in the coming year, and will remain 
as an instrument used by the Australian Government 
to determine the level of research funding available to 
Australian universities (Rowbotham 2010).While there 
has been some unease about the ERA amongst academ¬ 
ics, many seem resigned to the process. Perhaps some 
have simply accepted the onset of the audit regime 
and have bunkered down. Others perhaps welcome 
the chance to operate within the competitive environ¬ 
ment the ERA brings, having discarded (or perhaps 
never subscribed to) the older cultures of collegiality 
that, as we shall see, are hollowed out by cultures of 
audit. Others may simply believe that the ERA provides 
a relatively neutral way to measure and determine 
quality, thus accepting the benign, if somewhat unspe¬ 
cific assurances from Senator Kim Carr and Australian 
Research Council Chief Professor Margaret Sheil that 
academics who stick to what they are good at will be 
supported by the ERA. 

The ERA represents a full-scale transformation of 
Australian universities into a culture of audit. While 


aspects of auditing have been part of the Australian 
context for some time,Australian universities have not 
faced anything like say, the UK situation where inten¬ 
sive and complex research assessment exercises have 
been occurring for over two decades. Until now that is, 
and a glance at the state of higher education in the UK 
ought to give pause. Responding to the ERA requires 
more than tinkering with various criteria for measur¬ 
ing quality. Instead we suggest the need to return to 
basics’ and discuss how any comprehensive auditing 
regime threatens to alter and in all likelihood under¬ 
mine the capacity for universities to produce innova¬ 
tive research and critical thought. To say this is not to 
argue that these things will no longer exist, but that 
they will decline as careers, research decisions, cul¬ 
tures of academic debate and reading are distorted by 
the ERA. The essential ‘dysfunctionality’ of the ERA for 
institutions and individual researchers is the focus of 
this article. 

In discussing the pernicious impacts of auditing 
schemes we focus in particular on the journal ranking 
process that forms a significant part of the ERA. While 
the ERA will eventually rank other research activities 
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such as conferences, publishers and so on, the specif¬ 
ics of this process remain uncertain, while journals 
have been ranked and remain the focal point of discus¬ 
sions concerning the ERA. In what follows we explore 
the arbitrary nature of any attempt to rank’ journals, 
and examine the critiques levelled at both metrics and 
peer review criteria. We also question the assumption 
that audit systems are here to stay and the best option 
remains being attentive to the ‘gaps’ in techniques that 
measure academic research, redressing them where 
possible. Instead we explore how activities such as 
ranking journals are not only flawed but more signifi¬ 
cantly erode the very contexts that produce quality' 
research. We argue that collegiality, networks of inter¬ 
national research, the socio-cultural role of the aca¬ 
demic journal, as well as the way academics research 
in the digital era, are either ignored or negatively 
impacted upon by ranking exercises such as the ERA. 

As an alternative we suggest relocating the question of 
research quality outside of the auditing framework to a 
context once more governed by discourses of‘profes¬ 
sionalism’ and ‘scholarly autonomy’. 

In 2008 the Australian Labor Party introduced 
the ERA, replacing the previous government’s RQF 
(Research Quality Framework), a scheme that relied 
upon a fairly labour intensive process of peer review, 
the establishment of disciplinary clusters, panels 
of experts, extensive submission processes and the 
like. In an article entitled ‘A new ERA for Australian 
research quality assessment’ (Carr 2008), Senator Kim 
Carr argued that the old scheme was ‘cumbersome 
and resource greedy', that it ‘lacked transparency, 
and failed to win the confidence of the university 
sector’. Carr claimed that the ERA would be a more 
streamlined process that would reflect world’s best 
practice’. Arguing that Australia’s university research¬ 
ers are highly valued ... and highly respected’ Carr 
claimed that the ERA would enable researchers to be 
more recognised and have their achievements made 
more visible. 

If we took Senator Carr’s statements about the ERA 
at face value we would expect the following.The ERA 
would value Australian researchers by making their 
achievements more visible’. The ERA would reflect 
world’s best practice’ and reveal how Australian uni¬ 
versity researchers stack up against the best in the 
world’. Finally the ERA would gain the confidence of 
researchers by being a transparent process. All this 
would confer an appropriate degree of respect for 
what academics do. 
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‘Respecting Researchers’: the larger 
context that drives visibility 

According to Carr the ERA provides a measure of respect 
for academic researchers because it allows their work 
to be visible and thus measurable on the global stage. 
Given that academics already work via international col¬ 
laboration and publishers and processes of peer-review 
already embed value, the questions remains: for whom 
is this process of visibility intended? Arguably it is not 
intended for members of the academic community. 
Nor the university, at least in a more traditional guise, 
where academic merit was regulated via processes of 
hiring, tenure and promotion. In other words the idea of 
‘respect’ and ‘value’ already has a long history via institu¬ 
tional processes of symbolic recognition. 

Tying respect to the ERA subscribes to an altogether 
different understanding of value. Demanding that 
research be made more visible subscribes to a more 
general culture of auditing that has come to frame the 
activities of not merely universities but also schools, 
hospitals and other public institutions (Apple 2005; 
Strathern 1997). Leys defines auditing as ‘the use of 
business derived concepts of independent supervi¬ 
sion to measure and evaluate performance by public 
agencies and public employees’ (2003, p-70); Shore 
and Wright (1999) have observed how auditing and 
benchmarking measures have been central to the con¬ 
stitution of neoliberal reform within the university. 
Neoliberalism continually expects evidence of effi¬ 
cient activity, and only activity that can be measured 
counts as activity (Olssen & other forms of intellectual 
activity) that lies at the core of the ERA is not simply a 
process of identification or the reflection of an already- 
existing landscape, but rather part of a disciplinary 
technology specific to neoliberalism. 

The ERA moves away from embedded and implicit 
notions of value insisting that value is now overtly 
measurable. Outputs’ can then be placed within a 
competitive environment more akin to the commercial 
sector than a public institution. Michael Apple argues 
that behind the rhetoric of transparency and accuracy 
lies a dismissal of older understandings of value within 
public institutions. The result is a 

de-valuing of public goods and services... anything 
that is public is ‘bad’ and anything that is private 
is ‘good’. And anyone who works in these public 
institutions must be seen as inefficient and in need 
of the sobering facts of competition so that they 
work longer and harder (2005, p.15). 
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Two things can be said here. First, rather than simply 
‘reflect’ already existing activities, it is widely recog¬ 
nised that auditing regimes change the activities they 
seek to measure (Apple 2005; Redden 2008; Strathern 
1997). Second, rather than foster ‘respect’ for those 
working within public institutions, auditing regimes 
devalue the kinds of labour that have been historically 
recognised as important and valuable within public 
institutions. 

Outside of critiques that link auditing to a wider 
culture of neo-liberalism more specific concerns have 
been raised concerning the accuracy of auditing meas¬ 
ures. The degree to which any combination of statisti¬ 
cal metrics, peer or expert review, or a combination of 
both can accurately reflect 
what constitutes quality’ 
across a wide spectrum has 
been subject to critique 
(Butler 2007). With the 
ERA, concerns have already 
been raised as to the lack of 
transparency of the ranking 
process by both academics (Genoni & Haddow 2009) 
and administrators (Deans of Arts, Social Sciences and 
Humanities 2008). 

Though there is no universally recognised system 
in place for ranking academic journals, the process is 
generally carried out according to a process of peer- 
review, metrics or some combination of these meth¬ 
ods. The ERA follows this latter approach combining 
both metrics and a process of review by ‘experts in 
each discipline’ (Australian Research Council 2010; 
Carr 2008). Both metrics and peer review have been 
subject to widespread criticism. Peer review is often 
unreliable. There is evidence of low correlation 
between the reviewer’s evaluations of quality with 
later citations (Starbuck 2006,83-4).Amongst research¬ 
ers there is recognition of the randomness of some edi¬ 
torial selections (Starbuck 2006) with the result that 
reviewers are flooded with articles as part of a strategy 
of repeated submission. Consequently, many review¬ 
ers are overburdened and have little time to check the 
quality, methodology or data presented within each 
submitted article (Hamermesh 2007). In an early study 
of these processes, Mahoney (1977) found that review¬ 
ers were more critical of the methods used in papers 
contradicting mainstream opinion. 

The technical and methodological problems associ¬ 
ated with bibliometrics have also been criticised in the 
light of evidence of loss of citation data pertaining to 


specific articles (Moed 2002), as well as geographical 
and cultural bias in the ‘counting process’ (Kotiaho et 
al. 1999). Beyond this there are recognised methodo¬ 
logical shortcomings with journal ranking systems.The 
focus on journals, as opposed to other sources of publi¬ 
cation ignores the multiple ways scholarly information 
is disseminated in the contemporary era.The long time 
frame that surrounds journal publication, where up to 
three years delay between submission and publication 
is deemed acceptable, is ill-suited to a context where 
‘as the rate of societal change quickens, cycle times 
in academic publishing ... become crucial’(Adler & 
Harzing 2009 p.75). Citation counts, central to metri¬ 
cal systems of rank, do not guarantee the importance 
or influence of any one 
article. Simkin and Row- 
chowdhury’s (2005) analy¬ 
sis of misprints in citations 
suggest that 70 to 90 per 
cent of papers cited are not 
actually being read. Moreo¬ 
ver, there is no strong cor¬ 
relation between the impact factor of a journal and the 
quality of any article published in it (Adler & Harzing 
2009; Oswald 2007; Starbuck 2006). 

Neither peer review, nor metrics can accurately 
capture how academic research is carried out and dis¬ 
seminated. Nor do they provide guarantees of quality. 
However, as Adler and Harzing observe, the privileging 
of any combination of these measures leads to differ¬ 
ent material outcomes: 

Each choice leads to different outcomes, and thus 
the appearance - if not the reality of arbitrariness 
...whereas each system adds value within its own 
circumscribed domain, none constitutes an ade¬ 
quate basis for the important decisions universities 
make concerning hiring, promotion, tenure and 
grant making, or for the ranking of individuals and 
institutions (2009 pp.74-5). 

Senator Carr’s hope that the ERA would gain the 
trust’ of researchers is rendered problematic within 
a culture of audit. As Virno has observed ‘cynicism is 
connected with the chronic instability of forms of life 
and linguistic games’ (2004 p. 13). The move within 
Australia from the RQF to the ERA, the lack of transpar¬ 
ency as to the ranking process of journals within the 
ERA, the fact that there is no universal system of meas¬ 
urement, and that ranking bodies shuffle between the 
inadequate poles of metrics and peer-review, confirms 
the chronic instability of attempts to define and meas- 


With the ERA, concerns have already been 
raised as to the lack of transparency of the 
ranking process by both academics and 
administrators... 
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ure quality. The result can only be, at the very least, a 
distortion of research behaviour as academics recog¬ 
nise and cynically (or desperately) respond to quality 
measurement regimes. As we move from the RQF to 
the ERA with a change of government, the scope for 
‘chronic instability’ is vast. 

It is widely recognised that those subject to audit 
regimes change according to the perceived require¬ 
ments of the regime, rather than the long-held under¬ 
standing as to what intrinsic quality governs their work. 
Strathern (1997) and Power (1994) have persuasively 
argued that auditing regimes are not merely reflec¬ 
tive but are transformative. Such regimes contribute 
to the production of different subjectivities, with dif¬ 
ferent understandings and priorities Commenting on 
the reconstitutive capacity of auditing measures Cris 
Shore argues that audit has a life of its own - a runaway 
character that cannot be controlled. Once introduced 
into a new setting or context, it actively constructs (or 
colonises) that environment in order to render it audit- 
able’ (2008 p.292). 

Recognising the transformative nature of auditing 
allows us to focus on the unintended consequences of 
the journal ranking process. Privileging journal rank¬ 
ing as an indication of quality fails to comprehend how 
academics work within a contemporary context, how 
they work as individuals and as colleagues, how they 
co-operate across national and disciplinary borders, 
and how they research within a digital culture that is 
well on the way to displacing paper-based academic 
publishing. Indeed even if all the issues pertaining to 
accurate measurement, inclusion and transparency 
were somehow to be resolved, the ERA and the journal 
ranking exercise would remain at odds with the aim 
of generating sustainable quality research. Nowhere is 
this clearer than with the object at the heart of the 
process - the journal itself. 

Journal ranking and the transformation of 
journal publishing 

Why privilege the journal as the site for academic value? 
Beyond the problems in trying to measure journal qual¬ 
ity, the journal is undergoing a transformation. Journals 
are subject to a number of contradictory processes. On 
the one hand the journal as a place for disseminating 
research is partially undermined by alternative ways 
of circulating information. Adler and Harzing (2009) 
argue that academic research is no longer published 
just within the refereed journal but that books, book 


chapters, blog entries, conference papers and the like 
need to be taken as a whole as representative of con¬ 
temporary research culture . Moreover to place such 
a heavy evaluative burden on the journal, as the ERA 
does, fails to reflect the changed status and meaning 
of the journal within academic culture. Journal articles 
have become increasingly uncoupled from the jour¬ 
nal as a whole. The increasing centrality of electronic 
publishing means allows people to read individual 
articles rather than whole issues. In an observational 
study at three universities in Sweden, Haglund and 
Olsson (2008) found that researchers increasingly (and 
in many cases exclusively) rely on Google and other 
search engines for research information, bypassing 
libraries and traditional sources. 

Many researchers use a ‘trial and error’ method (2008 
p.55) for information searching, using a selection of 
keywords and evaluating the result.A flattening out of 
informational hierarchies results, where the content of 
individual articles becomes more significant than the 
journal that houses the articles. Electronic hyperlinks 
extend this shift where academic reading takes place 
beyond the pages of a (vertically ranked) individual 
journal to a horizontally network database of scholarly 
articles. This extends the trend identified by research¬ 
ers such as Starbuck (2006), whereby little correlation 
exists between articles and citation impact measured 
by journal quality. Ranking journals frames a mode of 
quality assessment around an increasingly irrelevant 
institutional form. 

Conversely the significance of a small number of 
journals has been enshrined through the auditing 
process. While academics know that there may be little 
correlation between the journal and the quality of indi¬ 
vidual articles, they also know that careers may now 
depend upon publishing in a journal whose value has 
been ‘confirmed’ by a process such as the ERA. In this 
sense, despite the decentring of journals via the infor¬ 
mation mode, the journal is destined to survive; some 
will flourish. However, this is hardly cause for cel¬ 
ebration given the general conservative approach to 
research taken by esteemed journals (Mahoney 1977), 
the knowledge that academics will tailor their work in 
order to fit in with the expectations of the journal in 
question (Reddon (2008) and finally, that many highly 
ranked journals are now products of transnational 
publishers, having long disappeared from the univer¬ 
sity departments that originally housed them and the 
community of scholars that sustained them (Cooper 
(2002, Hartley 2009). 
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This is not to dismiss the importance of the jour¬ 
nal, but to argue that journals are socio-cultural arte¬ 
facts whose most important work occurs outside of 
the auditing process. Ranking schemes like the ERA 
threaten to undermine the journal’s social and cul¬ 
tural importance. While journals are under threat by 
changes in publishing and digital modes of access 
and circulation, many continue to exist by reference 
to a (imagined and actual) community of readers and 
writers. The decision by a researcher to publish in a 
journal is often made in terms of the current topic 
being explored within the journal, the desire to dis¬ 
cuss and debate a body of knowledge already in that 
journal, invitations or requests by the editors, or calls 
for papers based upon a theme of interest to the aca¬ 
demic. In other words journal content or collegial 
networks frame decisions about where to publish as 
much as the perceived status of the journal (Cooper 
2002; Hartley 2009). 

The problem with rankings is that these relations 
are in danger of being overlaid by an arbitrarily com¬ 
petitive system so that scholars will no longer want, or 
be allowed to (by institutional imperative) publish in 
anything below a top ranked journal, as Guy Redden 
(2008) has observed with respect to the UK situation. 
We suggest that the transformative capacity of audit¬ 
ing measures such as the journal ranking scheme that 
constitutes the heart of the ERA threatens to produce a 
number of perverse or dysfunctional reactions within 
the academic community that threaten to undermine 
research quality in the long-term. 

The ERA and its perverse effect upon 
scholars and institutions 

Drawing on the above we want to focus specifically on 
some of the potential impacts of the journal ranking 
exercise. In particular, the potential for the mechanisms 
designed to measure ‘quality’ to create dysfunctional 
reactions and strategies within Australia’s research 
culture. Osterloh and Frey outline institutional and 
individual responses to research ranking systems, indi¬ 
cating that at the level of the individual, responses tend 
to follow the process of‘goal displacement’, whereby 
‘people maximise indicators that are easy to measure 
and disregard features that are hard to measure’ (2009 
p,12).As others have observed, the primacy of journal 
rankings in measuring quality for the Humanities runs 
a very high risk of producing such responses (Genoni 
& Haddow 2009; Nkomo 2009; Redden 2008). In his 


article published prior to the development of the ERA, 
Redden drew on his experiences of the UK’s Research 
Assessment Exercise (RAE) system, to observe that 
narrowly defined criteria for research excellence can 
result in ‘academics eschewjing] worthwhile kinds 
of work they are good at in order to conform’ (2008 
p. 12).There is a significant risk that a large proportion 
of academics will choose to play the game’, given the 
increasing managerial culture in Australian universi¬ 
ties and the introduction of performance manage¬ 
ment practices which emphasise short-term outputs 
(Redden 2008). 

In what follows, we attempt to flesh out the impact 
that the dysfunctionality introduced by the ERA will 
have on the research culture in the Humanities in 
Australia. These points are based on our observations, 
discussions with colleagues both nationally and inter¬ 
nationally, and review of the literature around research 
management systems. It is our argument that these 
impacts strike at the heart of collegiality, trust, the rela¬ 
tions between academics at different levels of experi¬ 
ence, how we find value in other colleagues, and how 
individuals manage their careers; all components fun¬ 
damental to research practice and culture.The ERA dis¬ 
places informal relations of trust and replaces them with 
externally situated forms of accountability that may well 
lead to greater mistrust and scepticism on the part of 
those subject to its auditing methods. This at least has 
been the experience of those subject to similar regimes 
in the UK (Power 1994; Strathern 1997). It should be 
noted that the potential for dysfunctional reactions has 
been acknowledged by both Professor Margaret Sheil, 
CEO of the Australian Research Council, and Profes¬ 
sor Graeme Turner, who headed the development of 
the ERA for the Humanities and Creative Arts clusters 
(McGilvray 2010, Rowbotham 2010). In both cases, uni¬ 
versities have been chastised for ‘misapplying’ the audit 
tool which, in Sheil’s words, “codified a behaviour that 
was there anyway” (Rowbotham 2010). 

Impact on international collaboration and 
innovation 

One impact of the ERA journal ranking system is the 
further complication it produces for international 
research collaboration. For many research practice is 
a globalised undertaking. The (limited) funds available 
for conference attendance, and the rise of discipline 
and sub-discipline based email lists and websites mean 
that many are networked within an internationalised 
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research culture in their area of specialisation. In the 
best case scenarios, researchers are developing con¬ 
nections and relationships with scholars from a range 
of countries. Before the ERA, these connections would 
form a useful synergy with a researcher’s Australian- 
based work, resulting in collaborations such as joint 
publications, collaborative research projects, and 
knowledge exchange. Such projects can now be the 
cause of significant tension and concern; an invitation 
from an international colleague to contribute an article 
to a low ranked (or heaven forbid, unranked) journal, 
to become engaged in a collaborative research project 
which results in a co-edited publication (currently not 
counted as research activity in the ERA), or to present 
at a prestigious conference must be judiciously evalu¬ 
ated by the Australian academic for its ability to ‘count’ 
in the ERA. This can be determined by consulting the 
ERA Discipline Matrices spreadsheet. Projects such as 
those listed above will need to be defended at the level 
of the individual’s performance management as the 
ERA is bedded down in performance management (a 
process which has already begun, with the discourse 
of the ERA being adapted internally by Australian uni¬ 
versities). These unnecessary barriers restrict open 
and free collaboration, as Australian researchers are 
cordoned off within a system which evaluates their 
research outputs by criteria which affects only Austral¬ 
ians. This seems even more perverse when we return 
to Senator Carr’s framing of the ERA process in global 
terms; seeing how Australian researchers ‘stack up 
against the rest of the world’ - that the ERA would 
represent world’s best practice’. Instead the structural 
provinciality built into a purely Australian set of rank¬ 
ings cuts across global research networks. 

In all likelihood, scholars will feel compelled to 
produce work that can be published in highly-ranked 
journals. The result of this is a new form of dysfunc- 
tionality; the distortion of research and its transfer. 
Redden argues that: 

Because of the valorising of certain kinds of output 
(single-authored work in prestigious form likely to 
impress an expert reviewer working in a specific 
disciplinary framework upon being speed read), 
researchers modify their behaviour to adapt to 
perceived demands. This means they may eschew 
worthwhile kinds of work they are good at in order 
to conform. Public intellectualism, collaboration, 
and interdisciplinary, highly specialised and teach¬ 
ing-related research are devalued (2008 p.12). 

If the ranking of journals narrows the possibility for 
innovative research to be published and recognised 


E R S I T I E S 

this situation may well be exacerbated by the uncer¬ 
tainty around new journals and emerging places of 
publication.The ERA seems unable to account for how 
new journals will be ranked, and arguably new jour¬ 
nals are a place where new and innovative research 
might be published. Yet, it takes a number of years for 
new journals to even be captured by the various met¬ 
rical schemes in place. For instance the ISI Social Sci¬ 
ence Citation Index has a three year waiting period 
for all new journals, followed by a further three year 
study period before any data on the journal’s impact is 
released (Adler & Harzing, 2009 p.80). Even for jour¬ 
nals ranked by alternate measures (such as Scopus) a 
reasonable period is required to gain sufficient data for 
the ranking of new journals. Such protracted timelines 
mean it is unlikely that researchers will gamble and 
place material in new journals. Equally the incentives 
to start new journals are undercut by the same pro¬ 
cess.The unintended consequence of the ERA ranking 
scheme is to foreclose the possibility of new and crea¬ 
tive research, and the outlets that could publish it. 

Impact on career planning 

Many early career researchers are currently seeking 
advice from senior colleagues on how to balance 
the tensions between the values of the ERA and their 
need to develop a standing in their field, especially in 
those discipline and sub-disciplines which have not 
had their journals advantageously ranked. The kind of 
advice being offered ranges from ‘don’t do anything 
that doesn’t count in the ERA’ to convoluted advice 
on how to spread one’s research output across a range 
of outcomes which cover both ERA requirements and 
the traditional indicators of quality associated with 
one’s area of specialisation. Professor Sheil has herself 
offered advice to younger academics, stating in a recent 
interview that: ‘You should get work published where 
you can and then aspire to better things’ (Robowtham 
2010). Within a year of the ERA process commenc¬ 
ing we already see evidence of academics being delib¬ 
erately encouraged to distort their research activity. 
McGilvray (2010) reports that scholars are being asked 
to switch the field of research they publish under if 
it will help achieve a higher future ERA rating . Jour¬ 
nalism academics at the University of Queensland and 
the University of Sydney have already switched their 
research classification from journalism to other catego¬ 
ries that contain more highly ranked journals. Similar 
examples are being cited in areas from cultural studies 
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to psychology. Such practices distort both the work of 
the researcher and threaten to further marginalise any 
journals contained within the abandoned field. Given 
the degree of institutional pressure it would be a brave 
researcher who would follow the ARC’S chief execu¬ 
tive Margaret Shell’s advice to focus on what you’re 
really good at regardless of where it is and that will win 
out’ (McGilvray 2010). 

While some senior academics (including Professor 
Sheil) are encouraging early career researchers to go 
on as though the ERA isn’t happening, and maintain 
faith that audit techniques will adequately codify the 
quality’ of their work, or at least retain confidence in 
the established practices of reputation and the power 
of the reference to secure career advancement, this 
remains a risky strategy. Others encourage a broader 
approach to publication, especially where a sub-dis¬ 
cipline’s journals have been inaccurately ranked, and 
advocate re-framing research for publication in highly 
ranked journals in areas such as Education. A genera¬ 
tion of early career researchers, then, are left to make 
ad hoc decisions about whether to value governmen¬ 
tal indicators or the established practices of their field 
with little understanding of how this will impact on 
their future prospects of employment or promotion. 

In her study of younger academics constructions 
of professional identity within UK universities, Archer 
noted a growing distance between older and newer 
generations of academics. Stark differences emerged in 
terms of expectations of productivity, what counted 
as quality research, whether managerial regimes ought 
to be resisted and so on. Evidence of intergenerational 
misunderstanding was found (2008 p.271) and while 
talk of academic tradition or a golden age’ prior to neo¬ 
liberalism was sometimes used to produce a bound¬ 
ary or place to resist managerialism, in many cases 
the discourse of older academics was resented or was 
regarded as challenging the authenticity of younger 
researchers. Instead of the idea of research and schol¬ 
arship as a culture to be reproduced, schemes such as 
the ERA threaten to drive a wedge between two very 
different academic subjectivities. 

Performance management by ranking leaves the 
individual academic in a situation where they must 
assiduously manage the narrowly-defined value of 
their publication practice and history (Nkomo 2009; 
Redden 2008). When the 2010 ERA journal rankings 
were released, many academics woke up to discover 
that their status as researchers had been radically re¬ 
valued (see Eltham 2010 for a blogged response to 
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this experience). Rather than contributing members 
of scholarly communities, individual researchers are 
now placed in direct competition with each other and 
must be prepared to give an account of their chosen 
publication venue in the context of performance man¬ 
agement and University-level collation of data for the 
ERA. So too the journals, and editors of journals, who 
will strive to increase the ranking of their publications 
at the necessary cost of others in their field.As Redden 
points out, such a situation runs the risk of importing 
the limits and failures of the market into the public 
sector (2008 p.16) as any re-ranking of journals will 
have direct effects on people’s employment. 

Lack of certainty about stability of 
rankings 

While researchers are left to make ad hoc decisions 
about their immediate and future plans for research 
dissemination, and ponder their ‘value’, they do so in 
an environment where there is no certainty about the 
stability of the current journal rankings. Given the long 
turnaround times of academic publishing it is increas¬ 
ingly difficult for people to feel confident that the deci¬ 
sions they make today about where to send an article 
will prove to be the right ones by the time they reach 
publication. Given the increase in submissions one 
expects A* and A ranked journals will receive, turna¬ 
round times are likely to increase rather than decrease 
with the introduction of the ERA. The erratic re-rank- 
ings that occurred between the last draff version of 
the journal rankings and the 2010 finalised list (where 
journals went from A* to C, with some disappearing 
altogether) have left many researchers uncertain as 
to whether current rankings will apply in 2012 when 
their article comes out. No one (not Deans of Arts, 
Social Sciences and Humanities, nor senior research¬ 
ers or other discipline bodies) seems able to provide 
certainty about the stability of the rankings, although 
many suspect that the current list will be “tweaked” 
in coming years. Again this has implications for career 
planning as well as internal accountability measures 
such as performance management, more importantly 
it unnecessarily destabilises the research culture by 
introducing the flux of market forces to evaluate what 
was traditionally approached as an open ended (or at 
least, ‘life’ (career) long) endeavour (see Nussbaum 
2010; Redden 2008). 

What is quality anyway? 
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Perhaps the most significant impact of attempts to 
quantify quality via a system of audit such as the ERA 
is that it works counter to the historical and cultural 
practices for determining quality that exist in academia. 
While these practices are in no way perfectly formed or 
without error, they do inform, sustain and perpetuate 
the production and distribution of knowledge within 
the sector internationally. As Butler has observed, any 
attempt to quantify quality via an audit system runs 
inexorably into the problem of how to define quality. 
Linda Butler, a leading scholar of research policy and 
bibliometrics, points out that research quality is, in the 
end, determined by the usefulness of a scholar's work 
to other scholars, and that quality' is a term given value 
socially (2007, p.568). She quotes Anthony van Raan 
who argues: 

Quality is a measure of the extent to which a group 
or an individual scientist contributes to the progress 
of our knowledge. In other words, the capacity to 
solve problems, to provide new insights into ‘real¬ 
ity’, or to make new technology possible. Ulti¬ 
mately, it is always the scientific community ('the 
peers’, but now as a much broader group of col- 
league-scientists than only the peers in a review 
committee) who will have to decide in an inter- 
subjective way about quality (van Raan (1996) in 
Butler, 2007 p.568). 

The Australian Research Council, in defending the 
ERA journal ranking for the Humanities and Creative 
Arts Cluster, relied heavily on this understanding of 
quality, citing the review panels, expert groups and dis¬ 
cipline representative bodies that were consulted in 
the determination of the rankings (ARC). Indeed, peer 
review and the sector’s involvement in determining 
what counts as ‘quality’ were central to Carr’s descrip¬ 
tion of the ERA (Carr 2008). However, and somewhat 
ironically given the audit culture’s obsession with 
accountability, the lack of available information regard¬ 
ing the debates about quality and its constitution 
which occurred in the formation of the list disconnect 
the concept of quality’ from its social, negotiated and 
debated context. As we have already noted, this lack 
of accountability does little to encourage academics 
to feel valued by the ERA process, nor does it support 
Australian academics in their existing practices of 
internationally networked research where the prevail¬ 
ing idea of quality, and how it is identified and assessed, 
is communal, collegial and plural. There is now, and 
will continue to be, a significant and unnecessary rift 
developing between international understandings of 
quality in research and the Australian definition. 
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Conclusion 

In the concluding chapter of The Audit Explosion , 
Michael Power diagnoses a key problem resulting from 
the rise of audit culture: we seem to have lost an abil¬ 
ity to be publicly sceptical about the fashion for audit 
and quality assurance; they appear as ‘natural solutions 
to the problems we face’ (1994 p.32). Many academ¬ 
ics remain privately sceptical about research auditing 
schemes but are unwilling to openly challenge them. 
As Power observed sixteen years ago, we lack the lan¬ 
guage to voice concerns about the audit culture’s focus 
on quality and performance (1994 p.33), despite the 
fact that in the Higher Education sector we have very 
strong professional and disciplinary understandings of 
how these terms relate to the work we do which are 
already benchmarked’ internationally. 

In light of this and the serious unintended outcomes 
which will stem from dysfunctional reactions to the 
ERA, we suggest that rather than try and lobby for small 
changes or tinker with the auditing mechanism (Aca¬ 
demics Australia 2008;AustralasianAssociation of Philos- 
ophy2008; Deans of Arts, Social Sciences and Humanities 
2008; Genoni & Haddow’s data 2009), that academics in 
the Humanities need to take ownership of their own 
positions and traditions around the idea of professional¬ 
ism and autonomy which inform existing understand¬ 
ings of research quality. Reclaiming these terms means 
not merely adopting a discourse of opposition or con¬ 
cern about the impact of procedures like the ERA (often 
placed alongside attempts to cooperate with the pro¬ 
cess) but adopting a stance that might more effectively 
contribute to the very outcomes of quality and innova¬ 
tion that ministers and governments (as well as academ¬ 
ics) desire. Power’s suggestion is that ‘concepts of trust 
and autonomy will need to be partially rehabilitated 
into managerial languages in some way’ (1994 p.33) and 
we may well begin with a task such as this. As Osterloh 
and Frey (2009) demonstrate, if academics are permit¬ 
ted to work informed by their professional motivations 
- intrinsic curiosity, symbolic recognition via collegial 
networks, employment and promotion - governments 
will be more likely to find innovation and research that, 
in Kim Carr’s words, you could be proud of’. 

Simon Cooper teaches in the School of Humanities, Com¬ 
munications & Social Sciences and Anna Poletti teaches 
in the School of English, Communications & Performance 
Studies at Monash University, Victoria, Australia. 
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